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PREFACE 


In  this  Memorandum  the  author  presents  a  number  of 
results  in  the  theory  of  finite  two— person  games — a  theory 
that  has  been  found  to  have  significant  applications  to  a 
variety  of  conflict  situations.  Much  of  the  material 
presented  here  has  been  derived  from  the  following  earlier 
Memoranda  (by  the  same  author):  RM-641,  EM-1142.,  RM-1145, 
RM-1598,  RM-2476,  and  RM-3026;  in  almost  every  case, 
however,  new  results  have  been  included. 


SUMMARY 


This  Memorandum  consists  of  several  loosely-related 
essays  on  the  theory  of  finite,  two— person  games.  The 
topics  covered  are,  in  brief,  (1)  the  block  decomposition 
of  symmetric  games,  (2)  saddlepoints  in  matrices  having 
submatrices  with  saddlepoints,  (3)  generalized  saddlepoints 
and  "order  matrices,"  (4)  the  existence  of  values  in  games 
with  almost— perfect  information,  and  (5)  the  non convergence 
of  "ficitious  play"  in  non— zero— sum  games.  Throughout, 
there  is  an  emphasis  on  features  of  the  theory  that  depend 
only  on  the  ordering  of  the  payoffs,  as  opposed  to  their 
numerical  values. 
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SOME  TOPICS  IN  TWO-PERSON  GAMES 


INTRODUCTION 

This  Memorandum  reports  on  half— a— dozen  loosely 
related  excursions  into  the  theory  of  finite ,  two— person 
games,  both  zero— sum  and  non— zero— sum.  .The  connecting 
thread  is  a  general  predilection  for  theorems  and  examples 
that  do  not  depend  upon  the  full  linear  structure  of  the 
real  numbers.  Thus,  many  of  the  results  we  present  are 
invariant  under  ordei>-preserving  transformations  applied 
to  the  payoff  spaces,  while  those  in  Sec.  1  are  invariant 
under  the  group  of  transformations  that  commute  with 
multiplication  by  —1. 

We  should  make  it  clear  that  we  are  not  studying 
"ordinal"  utility,  as  such,  but  rather  the  ordinal 
properties  of  "cardinal"  utility.  The  former  would 
demand  some  conceptual  reorientation,  which  we  do  not 
wish  to  undertake  here.  Nevertheless,  the  ordinalist  may 
find  useful  ideas  in  this  paper. 

We  shall  not  summarize  the  whole  paper  here,  but 
merely  sample  a  few  items;  for  a  more  synoptic  view  the 
reader  is  invited  to  consult  the  table  of  contents. 
Consider  first  the  matrix  game  shown  in  the 
right-hand  margin.  The  solution  is  easily 
found  as  soon  as  we  recognize  that  the  game 
is  symmetric  in  the  players.  Indeed,  if  we 
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map  each  player’s  i— th  strategy  into  the  other's  i+1— st  . 
(mod  4),  we  merely  reverse  the  signs  of  the  payoffs. 

It  folloxvs  that  the  value  is  0  and  that  there  is  a  solution 
of  the  form  (a,b,a,b),  (b,a,b,a).  (See  Sec.  1.) 

Next,  consider  the  class  of  matrix  games  in  which 
the  payoffs  are  ordered  like  those  in  the  matrix  at  left. 

Curiously,  in  all  these  games,  player  I's  third 
strategy  is  never  playable,  although  it  is  not 
dominated  in  the  usual  sense.  To  verify  this, 
observe  that  if  the  value  of  the  2— by— 2  subgame 
in  the  upper  left  corner  is  greater  than  "3"  the  third 
row  is  dominated  by  a  linear  combination  of  the  first  two 
rows,  while  if  it  is  less  than  "7"  the  third  column  can  be 
dropped,  and  then  the  third  row.  (See  Sec.  3.) 

Finally,  consider  the  non— zero— sum  game  with  outcome 
matrix  as  shown  at  right.  Player  I  rates  the 
outcomes  A  >  B  >  C;  player  II  rates  them 
B  >  A  >  C.  If  we  apply  the  algorithm  of 
"ficitious  play"  to  this  game,  a  strange  thing  happens. 
Rather  than  converging  to  the  unique  equilibrium  point 
(at  which  all  probabilities  are  equal),  the  sequence  of 
mixed— strategy  pairs  generated  by  the  algorithm  oscillates 
around  it,  keeping  a  finite  distance  away.  (See  Sec.  5.) 

The  five  main  sections  of  this  paper  are  essentially 
independent,  both  logically  and  topically.  Our  excuse 
for  putting  them  into  a  single  report  is  the  expectation 
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that  they  will  appeal  to  a  single  audience.  Much  of  this 
work  has  already  appeared  in  short  RAND  Memoranda  (see 
Ref.  [15]  at  the  end  of  the  paper),  and  some  of  it  has 
been  cited  in  the  published  literature  ([5], [11]).  In 
reworking  this  material,  however,  we  have  added  many  new 
results. 
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1.  SYMMETRIC  GAMES 
1.1.  Discussion 

It  is  easy  to  see  that  a  two— person  zero— sum  game 
can  be  symmetric  in  the  players  without  having  a  skew1- 
symmetric  payoff  matrix.  "Matching  Pennies"  is  a  simple 
example;  another  is  shown  at  the  right; 
another  is  given  in  the  Introduction. 

The  point  is*  of  course ,  that  an 
automorphism  of  the  game  that  permutes  the  players  can 
simultaneously  shuffle  the  labels  of  the  pure  strategies. 

It  would  be  interesting  to  know  something  about  the  abstract 
structure  of  such  automorphisms.* 

As  a  first  step,  we  shall  show  that  the  matrix  of  a 
symmetric  game  can  be  decomposed  into  an  array  of  square 
blocks  in  such  a  way  that  (a)  each  block  has  constant 

diagonals  (in  one  direction),  (b)  the  array 
as  a  whole  is  skew— symmetric  in  a  certain 
sense,  and  (c)  the  size  of  each  block  is  a 
power  of  2.  This  is  illustrated  at  left 
,  for  the  3— by— 3  example  given  above. 

The  "power  of  2"  property,  (c),  is  quite  interesting. 

It  tells  us,  for  example,  that  the  6-hy— 6  game  illustrated 
below,  which  is  obviously  symmetric  and  has  constant 

1  —  i  i 

The  narrow  "skew— symmetric"  definition  is  most  often 
seen  in  the  literature  (e.g.,  [19],  [10],  [7]).  But  Nash 
uses  the  more  general  form  in  [13] J  see  also  [19],  p.  166. 
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diagonals,  can  nevertheless  be  decomposed  into  smaller 
blocks.  The  4— by— 4  and  8— by— 8  analogues  of  this  matrix, 
on  the  other  hand,  do  not  decompose. 
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1.2.  The  Main  Theorem 

Let  A,  B,...  denote  n— by— n  game  matrices  (n  fixed 
throughout),  and  let  P,  Q,  R,...  denote  permutation 
matrices  of  the  same  size.  Primes  will  denote  transposition, 
which  is  equivalent  to  inversion  for  permutation  matrices. 

We  make  the  following  definitions: 

l 

equivalent :  A  =  B  <■-  ==>  A  =  PBQ1  for  some  P,  Q. 

symmetric :  A  e  2  <==>  A  s  -A 1 . 

conjugate:  P  #  Q  <===>  p  =  RQR1  for  some  R. 

Certain  subclasses  of  2  will  be  of  interest: 

A  e  2 (P,Q)  <==>  B  =  — PB'Q'  for  some  B  =“A'V 
These  subclasses  exhaust  2,  but  are  not  disjoint. 

Lemma  1.  If  PQ  RS  then  2(P,Q)  =  2(R,S). 

Proof.  Given  PQ  ==  TRST’  and  A  =  —  PA’Q*,  we  require 
B  s  A  such  that  B  =  -RB'S*.  As  it  happens,  B  =  T'AP'TR 
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serves  the  purpose.  Indeed,  since  P,=  QTS'R't',  wc.  have 
B  =  T'  (~PA'Q')  (QTS'R'T^TR 

=  -t'pa'ts' 

=  -(t'ap't) 's' 

=  -rb's'. 

Note  that  £(P,Q)  depends  only  on  PQ.  If  we  define 
£(P)  =  £(P,1),  where  I  is  the  identity,  then  Lemma  1 
may  be  restated: 

Lemma  1'.  If  P  «  Q  then  £(P)  =  £(Q ). 

It  can  be  shown  that  the  converse  is  valid.  Since 
two  permutations  are  conjugate  if  and  only  if  their  cyclic 
factors  have  matching  periods,  there  is  thus  a  one-to-one 
correspondence  between  the  classes  E(P)  and  the  partitions 
of  n.  This  stronger  result  is  not  needed  in  what  follows, 
however . 

Lemma  2.  If  Q  is  an  odd  power  of  P  then 
£(Q)  £  E(P). 

Proof.  Suppose  A  =  —PA1.  We  must  show  that 
A  e  2(P2k+1)  for  all  k  >  0.  But  A  =  ~P(-PA')'  =  PAP'j 
hence  A  -  PkAP,k  =  ~Pk+1A'P'k.  Therefore 
A  €  £(Pk+1,  Pk)  =  £(P?k+1). 

Theorem  1.1.  Every  symmetric  game  A  e  E  is 
equivalent  to  a  game  B  satisfying  B  =  — RB1  for  some 


permutation  R,  the  order  of  which  is  a  power  of  2. 
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Proof .  Let  A  e  £(P).  The  order  of  P  can  be 
represented  in  the  form  c2^}  with  c  odd.  Then  the 
order  of  Pc  will  be  2^.  By  Lemma  2,  Ae  E(PC).  Thus 
Pc  will  serve  as  the  R  of  the  theorem.  Q.E.D. 


1.3.  Block  Decomposition 

The  decomposition  into  blocks  can  now  be  described. 
By  proper  choice  of  B,  we  can  give  R  the  form: 

R  =  (1  2  3  ...  XjHXj+l  ...  X-J+  X2)  (...  n) , 

where  the  periods  X  are  powers  of  2.  B  now  breaks 
up  into  a  square  array  of  X  — by— X  blocks  B  ,  and 
the  equation  B  =  — RB*  implies  (and  is  implied  by)  the 
structure  set  forth  in  the  following  theorem. 


Theorem  1.2.  Let  X  =  min(X  ,  X  ).  Then  B  has 
constant  diagonals,  in  the  sense  that  numbers 
exist  such  that  for  TJ  in  that  blocks 


b^  if  i  ~  j  s  h(mod  X). 


(If  X^  =1=  Xv,  then 


5  breaks  up  into  identical  square 
sub— blocks  of  size  X . )  In  the  symmetrically  located 


block  B 


vp 

b.  .  = 
iJ 


the  same  numbers  appear;  we  have 
X— h+1  if  1  “  3  -  h(mod  X). 


In  particular,  along  the  main  diagonal  of  the  array  (|i  =  v), 
we  have 


Ph  "  “  ^X— h+1  * 
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Proof «  Theorem  1. 

Corollary  1«  Indecomposable  symmetric  n— by— n  games 
exist  only  for  n  a  power  of  2. 

Corollary  2.  Every  symmetric  game  of  odd  size  has 
a  zero  in  its  payoff  matrix. 

1.4.  Solutions 

To  solve  a  symmetric  game  we  (a)  replace  the  B^v 
by  their  average  values,  (b)  solve  the  resulting  skew— 

-ft 

symmetric  matrix,  and  (c)  distribute  the  mixed-strategy 
probabilities  for  each  block  equally  among  its  constituent 

-ft-ft 

pure  strategies.  Not  every  solution  of  the  original  can 
be  obtained  in  this  way,  however.  In  the  game  at  the 
right,  for  example,  (0,  2/3,  1/3,  0) 
is  one  of  the  basic  optimal  strategies 
of  each  player. 

1.5.  Symmetric  Nonzero— sum  Games 
There  is  a  direct  extension  to  nonzero— sum  games. 

Let  us  call  the  matrix  pairs  (A-^,  A2)  and  (BpI^) 

equivalent  if,  for  some  P,  Q,  both  A^  =  PB^Q'  and 

A2  =  PB2Q1.  Let  us  call  (A-^,  A2)  symmetric  if  (A-^,  A2) 

and  (A^,  A-p  are  equivalent.  Then  the  following  counterpart 

to  Theorem  1.1  can  be  established  by  essentially  the  same 

proof: 

— - 

^|>ee  Kaplansky  [10]  and  Gale,  Kuhn,  and  Tucker  [7]. 

See  Gale,  Kuhn,  and  Tucker  [8],  application  (e), 
pp.  94-94. 
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Theorem  1.3.  Every  symmetric  nonzero— sum  game 
(A p  A2)  is  equivalent  •  to  a  game  of  the  form  of  (B-^,  B2) 

=  (RB^j  RB-p  for  some  permutation  R  of  order  power  of  2. 

The  description  of  the  block  decomposition  remains 
much  as  before,  though  the  "main  diagonal"  loses  some  of 
its  special  significance.  Corollary  1  remains  valid,  but 
not  Corollary  2. 

2.  SOME  THEOREMS  ABOUT  SADDLEPOINTS 

2.1.  A  Condition  for  the  Existence  of  a  Saddlepoint 

Theorem  2.1.  If  A  is  the  matrix  of  a  zero— sum  two—,, 
person  game,  and  if  every  2— by— 2  submatrix  of  A  has  a 
saddlepoint,  then  A  has  a  saddlepoint. 

Proof.  Let  val[A]  =  v.  Let  j  be  the  index  of 
a  column  having  the  minimum  number,  tt,  of  entries  greater 
than  v.  Suppose  rr  >  0;  then,  for  some  i  we  have 
a—  >  v.  Then  we  must  have  a,  .  1  ^  v  for  some  j  1 .  But 
the  column  indexed  by  j 1  has  at  least  n  entries 
greater  than  v,  too  many  to  be  paired  off  against  the 
f— 1  remaining  entries  >  v  of  the  other  column.  Thus, 
for  some  i*  we  have  aiij»>  v  ^  a^.  Since  the 
2— by— 2  submatrix: 

i 
i 


j  j 


>  V 

£  V 

£  V 

>  V 
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clearly  has  no  saddlepoint,  the  assumption  tt  >  0  was 
incorrect.  Hence  there  is  a  column  with  no  entries 
greater  than  v.  Similarly  there  is  a  row  with  no  entries 
less  than  v.  Q.E.D. 

2.2.  Detached  Rows  and  Columns 

The  hypothesis  of  Theorem  2.1  actually  imposes  a 
very  special  structure  on  the  matrix  A.  Let  us  say 
that  the  p— th  row  of  A  is  detached  if 

max  a  .  <  max  min  a... 

j  PJ  "  ifp  j 

Similarly,  the  q— th  column  is  detached  if 
min  a .  >  min  max  a .  . . 

i  lq  ■  j+q  i 

Detachment  obviously  implies  domination.  For  2— by— 2 
matrices,  the  existence  of  a  detached  row  or  column  is 
equivalent  to  the  existence  of  a  saddlepoint. 

Theorem  2.2.  If  every  2— by— 2  submatrix  of  A  has 
a  saddlepoint,  then  A  has  a  detached  row  or  column. 

Pxflpf,.  By  Theorem  2.1,  both  A  and  A?  (its 
transpose)  have  saddlepoints.  Hence  there  is  a  column 
of  A  with  no  entries  greater  than  val[A],  as  well  as 
a  column  of  A  (row  of  A')  with  no  entries  less  than 
val[A'].  If  va3.[A]  <  val[A'],  then  these  columns  are 
distinct,  and  the  latter  is  a  detached  column.  Similarly, 


J 
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if  val[A]  >  val[A*],  there  is  a  detached  row.  If 
val[A]  =  val[A']  =  v,  then  there  is  either  a  detached 
row  or  column.,  or  a  saddlepoint  pq  common  to  both  A 
and  A'.  In  the  latter  case,  a^  s  =  v,  all  i,j, 
and  we  can  use  the  fact  that  the  submatrix  obtained  by- 
deleting  row  p  and  column  q  has  a  saddlepoint  to  show 
that  either  p  or  q  is  detached,  depending  on  whether 
the  value  of  the  submatrix  is  >  v  or  <  v.  Q.E;D. 

2.3.  A  Generalization 

Theorem  2.1  can  be  generalized,  after  a  fashion. 

Let  us  say  that  a  matrix  is  "in  general  position"  if  no 
two  collinear  entries  are  equal. 

Theorem  2.3.  Let  A  be  an  m— by— n  matrix  in  general 
position,  and  let  2  ^  r  <]  m,  2^s^n.  If  every  r— by— s 
submatrix  of  A  has  a  saddlepoint,  then  A  has  a 
saddlepoint. 

Proof.  It  suffices  to  prove  the  case  where 
r  =  m  ^  2  and  s  =  n— 1  2;  the  rest  will  follow  by 

induction  and  symmetry.  Let  A^  denote  A  with  the 
q— th  column  deleted.  Let  i^j^  the  location  of  the 
saddlepoint  of  A^  (which  is  unique,  since  A  is  in 
general  position).  If  all  of  the  j  's  are  distinct,  for 
q  =  l,...,n,  then  every  column  of  A  will  contain  one 


of  the  points  ij  .  Since  each  a.  . 

qJq  iqjp 


is  a  column 
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maximum,  one  of  them  will  be  the  maximum  of  the  whole 
matrix.  On  the  other  hand,  it  must  also  be  the  (unique) 
minimum  of  its  row  in  A^,  which  contains  at  least  two 
entries.  This  impossibility  implies  that  the  j  's  are 
not  all  distinct.  Let  j  =  jr,  q  f  r.  Then  it  is 
apparent  that  the  point  i'^'j ~  =  irj~  is  a  saddlepoint 
of  A.  Q.E.D. 

At  right  we  illustrate  what  can  happen  if  two 
collinear  entries  are  equal.  Every 
3„by— 2  submatrix  has  a  saddlepoint, 
but  not  the  full  matrix. 

3.  ORDER  MATRICES 

3.1.  Definitions:  The  Saddle 

By  a  line  of  a  matrix  we  shall  mean  either  a  row 

or  a  column.  Two  numerical  matrices  of  the  same  size 

will  be  called  order  equivalent  if  the  elements  of 

corresponding  lines  are  ordered  alike.  An  order  matrix, 

OL  ,  is  an  equivalence  class  of  order— equivalent 

numerical  matrices.  Abstractly,  it  may  be  regarded  as 

a  partial  ordering  <  on  the  set  I(£L)  of  all  index 

Cb 

pairs  ij ,  with  the  property  that  collinear  points  are 
always  comparable  while  noncollinear  points  are  never 
comparable,  except  as  required  by  transitivity. 

If  K  £  I (CV)  is  a  set  of  index  pairs,  we  shall 
write  K-^  for  the  set  of  first  members,  for  the  set 
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of  second  members,  and  K  for  x  K2»  In  other  words, 

K-^  is  the  smallest  set  of  rows,  K2  the  smallest  set 

of  columns,  and  K  the  smallest  submatrix,  that  covers 

K.  If  K  =  K  then  K  is  rectangular. 

A  generalized  saddle  point  (GSP)  of  an  order  matrix 

CL  is  a  rectangular  set  K  c  I(<Z)  such  that  (1)  for 

each  i  ^  Ki  there  is  a  p  e  K-,  with  pj  >  ij  for  all 
1  CO 

/  . 

j  e  K2,  and  (2)  for  each  j  f  K2  there  is  a  q  e  K2 
_  * 

with  iq  <  ij  for  all  i  e  K. .  (Note  the  strict 
Cb  1 

inequalities.)  A  GSP  that  contains  no  other  GSP  is 
called  a  saddle. 

Theorem  3.1.  Every  order  matrix  has  a  unique  saddle 

The  proof  consists  in  showing  that  the  intersection 

of  two  GSP' s  is  a  GSP.  Let  K  and  L  be  two  GSPs  of 

CL  .  Then  certainly  K  fl  L  4=  0,  since  K  and  L  must 

both  carry  the  optimal  mixed  strategies  of  any  numerical 

matrix  belonging  to  CL.  It  will  suffice  to  show  that: 

for  each  i  £  there  is  a  p  in  fl  L-^  with 

pj  >  ij  for  all  j  e  K,  fl  L9.  But  we  can  find  a 
d,  l  l 

p'  e  K-^  with  that  property,  since  K  is  a  GSP.  If 

p'  £  L, ,  then  we  can  find  a  p"  e  L-,  with  p"j  >  p'j 

1  CL 

for  all  j  e  K2  fl  L2»  If  p"  £  then  p*'*  e  K-^  can 

—  n 

Compare  [5],  p.  41—42. 

Bass  in  [1]  showed  how  to  prove  that  K  (1  L  f  0 
without  making  use  of  the  concept  of  optimal  strategies. 
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be  found  bearing  the  same  relation  to  p",  etc.  The 
strict  inequalities  ensure  that  the  sequence  terminates. 

That  is,  there  is  a  row  in  fl  L-^  that  majorizes  row 
i  on  K£  H  L^.  Q.E.D. 

The  saddle  of  CL  will  be  denoted  by  S (Cl)  •  It 
contains  everything  relevant  to  the  solutions  of  the 
numerical  games  A  belonging  to  CL  .  In  particular, 
it  contains  Bohnenblust Ts  "essential"  submatrix  and  the 

-ft"* 

Shapley— Snow  "kernels . " 

We  may  define  a  weak  GSP  by  using  nonstrict  inequalities 
^  ^  .  The  intersection  of  two  weak  GSP's  need  not  be  a 

a,  cl 

weak  GSP.  However,  as  Bass  has  shown,  the  intersection 
of  a  weak  GSP  with  the  saddle  is  a  weak  GSP. 

3.2.  Residuals 

A  one— element  saddle  is  an  ordinary  (strict)  saddlepoint, 

and  is  easy  to  find  even  in  a  very  large  matrix.  Identification 

of  the  saddle  in  general  may  be  more  tedious.  The  next 

*** 

theorem,  based  on  an  ingenious  idea  of  Harlan  Mills, 
provides  a  rapid  method  of  generating  points  in  the 
saddle,  and  thereby  reduces  the  search  problem  substantially. 

Lemma  1.  (Mills)  Let  CL,  be  an  order  matrix  with  at 
least  two  columns,  let  pq  be  maximal  with  respect  to 
<  ,  and  let  be  obtained  from  CL  by  deleting  column  q. 

Then  S(£q)  5  S(&). 

*See  [2],  p.  52;  also  [ 9 J ,  p.  44. 

See  [17],  p.  32. 

In  conversation. 
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Proof.  Write  S  for  S(Cb )  and  for 

S-^  x  (S2—  {q}).  If  q  £  S2>  then  is  at  once  seen 

to  be  a  GSP  of  CL0*-.  If  q  6  S2,  then  p  e  by 
maximality.  To  show  that  is  a  GSP  of  OC*- ,  note  that 

the  row  condition  is  satisfied  for  S^-,  just  as  for  S, 
while  the  column  condition  fails  only  if  column  q  was 
used  (in  CL  )  to  minorize  some  column  outside  S2»  But 
this  is  impossible,  by  the  maximality  of  pq.  Thus 
is  always  a  GSP  of  CL0*-.  Hence  S(<^-^)  c  c  s. 

A  similar  lemma  holds  concerning  the  deletion  of 
rows  containing  minimal  elements. 

The  following  lemma  is  easily  established}  we  omit 
the  proof. 

Lemma  2.  If  J3  is  obtained  from  CL  by  deletion  of 
a  strictly  majorized  row  or  a  strictly  minorized  column, 
then  S(w0  )  5  S(&). 

Now  consider  a  sequence  of  nonvoid  order  matrices 
^,a.(1),  a(2),...,  each  obtained  from  the  preceding 

by  deleting  a  column  containing  a  maximal  point,  or  a 
row  containing  a  minimal  point,  or.  a  strictly  majorized 
row,  or  a  strictly  minorized  column.  If  LL  is  an  m— by— n 
matrix,  then  ^  is  1— by— 1.  The  point  so  defined 

is  called  a  residual  of  CL.  Let  R (LL)  denote  the  set 
of  all  residuals  of  (L  .  Our  lemmas,  together  with  the 
fact  that  the  saddle  is  rectangular,  give  us  the  following 
result : 
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Theorem  3.2.  R(&)  <=  R (0.)  c  S(&). 

3.3.  Examples 

Let  us  illustrate  the  residual  concept. 

(a)  In  the  order  matrix  of  "Matching  Pennies" 

(at  right),  every  entry  is  either  maximal  or 
minimal.  By  the  maximality  of  11  we  can  strike¬ 
out  the  first  column,  and  then  the  first  row, 
and  obtain  22  as  a  residual.  The  three  other  possible 
sequences  give  us  TT,  T2",  and  71  as  residuals.  Thus 
R(<£)  =  RT^T  =  S(GL)  =  I  (CL). 

(b)  More  generally,  let  Cb  be  a  square  matrix  with 
a  "detached"  diagonal;  for  example,  let  TT  be  maximal 
for  all  i..  Then  for  each  i,  we  can  delete  all  columns 
except  i  by  maximality,  and  then  by  row  deletions  obtain 
ii  as  a  residual.  Hence  R(Tt)  =  S (CL)  ~  I(<Z). 

(c)  We  first  conjectured  [15b]  that  deletions  of 
the  first  type  (Lemma  1)  would  always  suffice  to  determine 
the  saddle.  This  was  disproved  by  Bass  [1]  with  the  4— by—4 
example  at  right,  which  has  no  residuals 
"of  the  first  type"  in  the  top  row.  But 
if,  for  example,  we  strike  out  column  3, 
then  row  3  goes  by  domination  (Lemma  2), 
and  the  resulting  3— by— 3  game  has  a  detached  diagonal, 
making  lT  a  residual  by  (b)  above.  (In  the  diagram  we 
have  circled  all  residuals.) 


1  — 1 
-1  1 


(d)  Thus,  a  new  counterexample  is  needed  to  show 
that  R (d)  f  S(A)  is  possible.  It  is  provided  by  the 
example  in  the  Introduction,  which  we  have  reproduced 
here  with  residuals  circled.  As  we  shall 
see  later,  it  is  no  coincidence  that  the 
residuals  avoid  the  "unplayable"  third  row. 


3.4.  Minimax  and  Maximin  Points 

A  minimax  point  of  a  numerical  matrix  is  a  column 
maximum  of  smallest  value.  A  maximin  point  is  a  row 
minimum  of  largest  value.  Since  noncollinear  comparisons 
may  be  involved,  these  terms  are  usually  not  well  defined 
for  order  matrices.  Nevertheless,  we  have  the  following: 

Theorem  3.3.  The  minimax  and  maximin  points  of  any 
numerical  matrix  A  e  <X  are  residuals  of  d,  . 

Proof.  Let  pq  be  a  minimax  point  of  some  A  e  d>  . 

If  A  has  more  than  one  column,  then  it  will  have  a 
maximum  in  some  column  other  than  q,  and  the  corresponding 
location  in  CL  will  be  maximal.  Delete  this  column  and 
pq  is  still  minimax.  Thus  we  may  strip  the  matrix  down 
to  just  the  single  column  q,  and  then  by  row  deletions 
obtain  pq  as  a  residual.  The  proof  for  maximin  points 
is  similar. 

Bass  [1]  has  shown  that  every  weak  GSP  contains  a 
minimax  point  and  a  maximin  point. 
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3.5.  An  Algorithm 

The  following  method  for  determining  the  saddle  was 
suggested  by  Mills  (see  [1],  p.  3):  Start  with  any 
submatrix  known  to  be  contained  in  the  saddle  (for 

example,  a  single  residual).  For  h  =  0,  1,...,  construct 

Kh+! 

from  K*1  by  adjoining  either  (a)  a  row  outside 

K*?  that  is  not  strictly  majorized  on  by  any  other 

row  of  CL  ,  or  (b)  a  column  outside  that  is  not 

strictly  minorized  on  by  any  other  column  of  CL  . 

h+l 

Then  it  is  easily  seen  that  K  c  S((2).  If  neither 
(a)  nor  (b)  is  possible,  then  Kh  =  S(CL). 

3.6.  The  Center 

Let  C-^(A)  denote  the  set  of  "active"  strategies 
for  the  first  player  in  A,  i.e.,  those  that  appear  with 
positive  probability  in  some  solution.  Similarly,  02(A) 
for  the  second  player.  Let  C(A)  =  C^(A)  X  02(A).  Then 
the  set 

C  (&)  =  U  C(A) 

A  eCL 

* 

will  be  called  the  center  of  CL.  The 
center  is  contained  in  the  saddle.  It 
is  not  necessarily  rectangular,  as  may 
be  seen  from  the  example  at  the  right 
(center  shaded) . 

- ^ - 

C(A)  is  the.  "essential"  submatrix  of  A  in  the 
sense  of  [9]  and  [2], 


Example  (d)  of  3.3  suggests  the  proposition  that  a 
point  that  never  occurs  in  optimal  play  cannot  be  a 
residual,  i.e.,  R (<Z)  c  C((L).  This  is  not  quite  true, 

as  the  example  at  left  shows  (center  shaded, 
residuals  circled).  The  equality  of  the  two 
1  *' s  accounts  for  this  exception,  as  the  next  theorem  shows* 

Theorem  3.4.  If  &  is  in  general  position,  then 
R (d  )  c  C (CL). 

(We  recall  that  CL  is  "in  general  position"  if  no  two 
collinear  elements  are  equal.)  The  rest  of  this  section 
will  be  devoted  to  the  proof  of  Theorem  3.4. 

Lemma  3.  If  pq;  e  C(A),  then  every  neighborhood  of 
A  contains  a  B  having  a  unique  solution  and  such  that 
pq  e  C  (B) . 

Proof.  If  pq  e  C(A)  then  a  "basic"  (extremal) 
solution  (x°,  y°)  of  A  exists  with  x°  >  0,  y°  >  0. 

We  may  assume  that  val[A]  =  v  0.  By  the  Shapley— Snow 
theorem  on  basic  solutions  [17],  a  nonsingular  submatrix 
A  can  be  found  for  which  (x°,  y°)  is  an  "equalizer": 

.Ox  *Oi  j  >i 

x  A  =  y  A  =  vl. 

(Dots  indicate  suppression  of  indices  not  appearing  in 

t  * 

A;  1  is  a  vector  of  l’s.)  Given  e  >  0,  define 
B  =  A  —  (e/v)A  .  If  e  is  sufficiently  small,  we  can 
invert  B,  as  follows: 


©CD 

© 
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B  1  -  A  1  +  (e/v)i  +  0  (e2) . 

Then  B  has  an  equalizer  solution  proportional  to  (x°,  y°)  + 

♦  •  o 

e(l,  1)  +  0(e  ).  This  is  all— positive  for  small  e,  and 
* 

hence  unique. 

We  now  replace  A  by  B  in  the  original  matrix.  Since 

* 

the  value  and  solution  of  B  differ  only  slightly  from  those 

♦ 

of  A ,  it  is  easy  to  perturb  the  rows  and  columns  outside 
the  submatrix  so  as  to  make  them  irrelevant.  The  full  matrix 
now  has  a  unique  solution,  and  p  and  q  are  still  active. 

Q.E.D. 

Lemma  4.  Let  A  have  at  least  two  columns,  let 
A^  result  from  A  by  deleting  column  q,  and  let  A  (id) 
result  from  A  by  adding  id  to  the  entry  at  pq.  Then 

lim  val[A(u;)]  =  val[A^]. 

UU-wo 

Proof.  Let  x°  be  optimal  in  A^.  Given  e  >  0, 
choose  x  such  that  jjx  —  x°||  _<  e  and  >  0.  Then 


xA(cd)  j>  (v-ea,  v— ea,...,— ; a+oux  , . . .  ,v—  ea), 

*  J 


where  v  =  val[A^l  and  a  =  max.. [a.. 

ij  1  xj 


(q— th  component) 

Thus,  for  id  large, 

x  guarantees  the  first  player  at  least  v  —  ea  in  the  game 

A  (id).  Hence  we  have 

v  —  ea  <  val[A(uj)]  <  v.  Q.E.D. 


Another  B,  suggested  by  0.  Gross,  is  given  by 
b..  =  a..  —  S.x.a. .  —  E.y.  a..  +  2  E.E.x.y.a.., 

ij  ij  ii  ij  J-Qj  ij  l  j  i'j  ij ' 


with  (x,  y)  all— positive  but  close  to  (x°,  y°). 


-22 


Lemma  5.  Let  . CL  be  in  general  position,  let  pq 
be  maximal  in  CL  ,  and  let  ofi  result  from  CL  by 
deleting  column  q.  Then  C(diq)  c  C(<&)* 

Proof,  Take  rs  in  the  center  of  (2q.  Using 
Lemma  3  and  the  fact  that  (L  is  in  general  position,  find 
A  e  CL  such  that  Aq  has  a  unique  solution  (x  ,  y*)  with 

•X-  -Jf 

x  >  0  and  y  >  0.  Define  the  functions 

L  S 

f(x)  =  min  2.  a . . x . ,  g(y)  =  max  2  a.,  y . . 

jfq  J  i  jk  J  J 

Since  f(x)  is  polyhedral  (piecewise  linear)  and  has  a 

* 

unique  maximum  at  x  ,  there  exists  a  >  0  such  that 
(3.1)  f(x)  <  val[Aq]  —  a  ||x  —  x|| 

for  all  mixed— strategy  vectors  x.  Similarly,  there  exists 
P  >  0  such  that 

(3*2)  g(y)  ^  val[Aq]  +  P  ||y*  -  y|| 


for  all  mixed— strategy  vectors  y  with  y^  =*  0. 

Now  let  A  (id)  be  obtained  from  A  by  adding  tu  >  0 
to  the  element  a  ,  and  let  (x(id),  y(iu))  be  any  solution 
of  A (u)).  Then  certainly  val[A(uj)]  <,  f(x(tu)),  and  we 
have  by  (3.1) 


a||x  —x (uj)  ||  ^  val[Aq]  —  val[A(tu)]* 


By  Lemma  4,  x(uj)  “•  x  as  id  -*  hence  xr(iu)  is  positive 
for  large  id. 
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The  corresponding  statement  for  y  (<u)  is  a  bit 

s 

harder  to  prove.  First  we  observe  that 

Ejapjyj  ^  +  “yqC1")  £  val  A(u)>  £  a. 


where  a  =  max^j | a^j | .  Hence 
yqO)  £  2a/ id  . 

Then  we  have 


val[A(u>)J  =  maxi  2b  aij  (tu)yj  (“»)  ^  2jaijyj  0") 

■  <»>  +  aiSyq("->  +  Kq-  als>yq<“>i 


-  ^ijl/ij^  (tU)  +  ais7q “  2ayq(w) 

^  g(y'  («>))  -  4a2/m, 


where  yT  (tu )  is  like  y(ou)  except  for  y'q((«)  =  0  and 
y'sW  =  ysO)  +y  (<u).  Hence,  by  (3.2), 

val[A(<u)]  ^  val[Aq]  +  P  ||  y*  —  y 1  (uu)  ||  —  4a2/uy, 

....  * 

and  y  (uj)  converges  to  y  ,  by  Lemma  4.  Since  y  (uj)  -*  0, 

*  ^ 
we  have  y  (iu)  -*  yo  >  0.  Thus  both  r  and  s  are  active 

o  o 

in  A  (in)  for  large  uj.  But  A(cu)e  CZ  for  all  positive  ui. 
Hence  rs  e  C(<&).  Q.E.D. 


Lemma  6.  If  $  is  obtained  from  by  the  deletion 
of  a  strictly  majorized  row  or  a  strictly  minorized  column. 


then  C(&  )  c  C(tf,) 
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Proof.  A  strictly  dominated  strategy  can  never  be 
active. 

Theorem  3.4  now  follows  directly  from  Lemma  5  (and 
its  "row"  counterpart).  Lemma  6,  and  the  definition  of 
"residual. " 

Equality  need  not  hold  in  Theorem  3.4;  for  example, 
in  (d)  above  we  have  21  e  C(dL)  —  R(CL ).  But  no  general- 
position  counterexample  has  been  found  to  the  proposition 

R  (0,  )  =  'CTSTJ- 

3.7.  The  Antisaddle 

We  close  with  a  simple  result  concerning  antisaddlepoints, 
i.e.,  saddlepoints  of  the  negative  matrix. 

Theorem  3.5.  A  strict  antisaddlepoint  of  CL 
cannot  be  in  the  center,  unless  CL  is  1— by— 1. 

Proof.  Suppose  S (—CL)  =  {pq}  S  C({2)»  Take  A  e  (JU 
such  that  "pq  e  C(A),  and  let  A  be  the  submatrix  of  A 

defined  by  C(A).  Then  A  has  an  equalizer  solution, 

*  • 

which  solves  —A  as  well.  But  —A  has  a  strict  saddlepoint 

at  pq1,  and  hence  a  unique,  pure— strategy  solution. 

* 

Therefore  A  is  1— by— 1  and  pq  is  a  saddlepoint  of  A 
as  well  as  of  —A,  strict  in  both  cases.  This  is  possible 
only  if  A  is  1— by— 1.  Q.E.D. 
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Weak  antisaddlepoints  are  sometimes  found  in  the 
center,  as  shown  (starred)  in  the  accompanying  examples. 


19 

The  smaller  matrix  has  a 

-i 

0  2 

saddlepoint;  the  larger  one 

(which 

is  symmetric  in  the  sense  of  Sec.  1) 

i 

i  -i  -i 

0 

does  not. 


Similar  reasoning  can  be  used  to  establish  the 
following  more  general  result,  which  applies  however  only 
to  numerical  matrices: 

Theorem  3.6.  If  S(A)  c  C(~A)  then  S(A)  =  C(-A). 

4.  INSTANTANEOUS  GAMES 

4.1.  Games  of  Almost— perfect  Information 

There  is  a  class  of  games  of  timing,  typified  by  the 
so-called  "noisy  duels"  (see  [5],  p.  128—134;  also  [11], 
Vol.  II),  in  which  the  payoff  depends  discontinuous ly 
upon  the  order  in  which  certain  actions  occur,  but  is  a 
continuous  function  of  the  occurrence  times,  given  the 
sequence  of  events.  The  (finite)  set  of  actions  available 
to  a  player  at  any  moment  may  depend  on  what  has  gone 
before;  in  any  case  the  players  are  informed  at  all  times 
of  all  previous  history.  But  for  the  possibility  of 
simultaneous  action,  a  solution  in  pure  strategies  would 
thus  be  indicated.  This  possibility,  however,  not  only 
makes  mixed  strategies  necessary  in  general,  but  even 
affects  the  existence  of  a  value.  Indeed,  there  are 
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some  surprisingly  simple  indeterminate  (valueless)  games 
of  this  type,  perhaps  worthy  of  a  place  beside  the  examples 
of  Ville  ([5],  p.  115)  and  of  Sion  and  Wolfe  ([18]). 

(See  below.  Fig.  1.) 

Simultaneity  is  not  a  serious  problem  in  the  middle 
of  the  game,  since  either  player  can  avoid  it  at  negligible 
cost  by  altering  his  timing  by  a  small  random  amount.  But 
this  tactic  may  not  work  at  the  beginning  or  end  of  the 
game,  since  the  random  displacement  would  be  in  a  fixed 
direction.  In  order  to  focus  on  the  problem  of  determi— 
nateness,  in  games  of  timing  with  almost— perfect  information, 
we  shall  assume  that  there  is  a  "critical  instant"  at  the 
beginning  of  the  game.  The  players  will  be  motivated 
(by  the  payoffs)  to  take  some  action  either  at  t  =  0 
or  immediately  thereafter.  We  thereby  reduce  the  infinite 
game  in  extensive  form  to  an  essentially  finite,  "instantaneous" 
game,  whose  payoffs  are  defined  by  the  values  of  the  subgames 
that  result  after  the  first  action,  or  pair  of  simultaneous 
actions. 

One  of  our  results  (Theorem  4.4)  states  that  an 
instantaneous  game  has  a  value  provided  that  the  payoff 
in  the  event  of  simultaneous  actions  always  lies  between 
the  payoffs  for  the  same  actions  performed  singly.  Another 
result  (Theorem  4.5)  shows  how  to  assign  a  formal  value 
to  the  game  when  the  true  value  does  not  exist. 
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4.2.  Some  Indeterminate  Noisy  Duels 

Two  gunfighters  are  crouched  behind  barriers,  where 
they  can  neither  see  nor  be  seen.  If  they  stand  up  at 
the  same  instant,  we  may  assume  that  their  chances  are 
equal  if  they  both  shoot  at  once,  or  if  they  both  hold 
their  fire.  If  only  one  fires,  that  one  wins.  If  one 
stands  up  without  shooting,  and  finds  the  other  still 
down,  then  he  can  take  control  of  the  situation,  and 
eventually  win.  But  if  the  one  under  cover  hears  the 
other  stand  up  shooting,  then  he  can  step  out  and  take 
control  before  his  opponent  has  time  to  reload. 

This  (with  apologies  to  the  real  world.')  is  an  example 
of  an  indeterminate  noisy  duel.  The  critical  moment  is 
at  the  beginning  of  the  time  interval,  since  to  wait  an 
appreciable  amount  of  time  is  definitely  bad  strategy. 

The  situation  at  t  -  0  can  be  represented  by  the 
following  matrix: 


Stand  up\ 
Shooting/ 

Stand  upl 
Quiet  / 

Wait 


Stand  up  Stand  up  Wait 
Shooting  Quiet _ 


0 

+1 

-1 

-1 

0 

+1 

+1 

-1 

® 

Here  "(r) "  is  meant  to  suggest  "repeat"  or  "replay,"  as 
in  a  recursive  game  (see  Sec.  4.4  below). 


Let  us  test  for  a  value.  In  the  minorant  game. 


in  which  player  I  openly  commits  himself  to  a  particular 
(mixed)  strategy,  the  outcome  (r)  is  worth  no  more  than 
— 1,  because  player  II,  knowing  the  probable  duration  of 
the  other's  "wait,"  will  almost  always  succeed  in  acting 
first.  Setting  (^R)  =  —1  and  solving,  we  find  that  the 
value  is  —1/9.  By  symmetry,  the  value  of  the  ma.j orant 
game  is  +1/9.  Hence  the  duel  is  indeterminate. 

An  even  simpler  example  is  shown  at  right.  The 
minorant  value  is  zero;  the  maj orant  value  is 
1/3.  Any  matrix  order— equivalent  to  this  one 
(see  Sec.  3)  also  represents  a  game  without  a 
value  if  we  interpret  "(r) "  as  a  number  between  0  and  1 
in  determining  order  equivalence. 

The  above  can  be  viewed  as  a  game  on  two  unit  squares, 
with  discontinuities  along  the  diagonals  (Fig.  1). 


1 

-1 

-1 

1 

0 

Figure  1. 
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4.3.  The  General  Case 

Let  M(cp)  denote  the  following  m+1— by— n+1  array: 


"act"  "wait" 


all  “**  aln 

a10 

"act" 

i  m 

m  • 

• 

♦ 

aml  amn 

am0 

"wait" 

a01  a0n 

cp 

Here  cp  is  either  a  real  number  or  the  symbol  (r)  .  The 
game  represented  by  M((R))  is  said  to  be  instantaneous 
if 

(4.1)  min  an.  <  max  a.n. 

jfO  °J  i=f=0  10 

This  condition  puts  a  premium  on  immediate  action.  Without 
it,  at  least  one  player  would  be  willing  to  delay,  and 
other  elements  of  the  original  game  of  timing  (from  which 
we  assume  the  data  in  M((r) )  to  have  been  drawn)  would 
become  significant  in  the  selection  and  timing  of  the 
first  action.  We  shall  be  concerned  henceforth  only  with 
the  instantaneous  case. 

The  minorant  and  majorant  games  have  the  matrices 

M,  =  M(min  an.)  and  M9  =  M(max  a.n), 
j+0  -1  z  ifo  lU 

respectively.  Let  their  values  be  v-^  and  and  their 

sets  of  optimal  strategies  X-^  and  X2  (for  player  I) 
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and  Y^  and  Y£  (for  player  II).  Obviously, 

(4.2)  min  aA.  <  <  v0  <  max  a.A. 

jfo  ~  1  “  2  -  i^O  10 

If  v-^  <  v 2  the  game  is  indeterminate;  if  =  V'2  the 
game  is  determinate  and  the  optimal  strategies  for  the 
two  players  correspond  exactly  to  the  elements  of  X-^  and 
Y2,  respectively. 

Theorem  4.1.  A  necessary  and  sufficient  condition 
for  the  instantaneous  game  M((R))  to  be  determinate  is 
that  X-^  5  X2  and  Y'2  c  Y^. 

Proof.  Necessity.  Any  strategy  guaranteeing  v^  to 
player  I  in  M-^  will  guarantee  him  at  least  as  much  in 
M2.  Therefore,  v-^  =  v2  implies  X^  5  X2.  Similarly, 

Y2  £  Yr 

Sufficiency.  Let  X-^  ^  X2>  Y2  fE  anc*  suPPose 
V1  ^  v2*  Take  x  e  X-^.  Then,  since  x  is  optimal  in 
M2 }  we  have 
m 

tfo  ^  -  Vz  >Vl'  J  ' 

Hence  no  j  =j=  0  can  be  active  in  M-^.  Hence  Y^ 
consists  of  the  single  strategy  "wait."  The  same  holds 
for  Y2>  by  inclusion,  and  for  X2  and  X^>  by 
symmetry.  Hence  M-^  and  M2  have  saddlepoints  at 
"wait— wait, "  contradicting  (4.1).  Q.E.D. 
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Theorem  4.2.  A  necessary  condition  for  the 
instantaneous  game  M((r))  to  be  determinate  is  that 
"wait"  is  unplayable  (i.e.,  not  active)  for  at  least  one 
player  in  at  least  one  of  M-^ ,  M2 . 

Proof.  Let  =  v2,  and  take  x  e  X^,  y  e  Y^- 

Then  x  e  X2  and  y  e  Y-^,  by  Theorem  4.1,  and  we  have 

0  =  v2  -  vi  =  ^y  -  xMiy  =  x0yQ(max  aiQ  -  min  aQ . ) . 

By  (4.1),  =  Since  x  and  y  may  be  chosen 

independently,  it  follows  that  either  Xq  vanishes 
identically  for  x  e  X^,  or  y q  vanishes  identically  for 
y  e  Y2-  Q.E.D. 

Theorem  4,3.  A  sufficient  condition  for  the 
instantaneous  game  M((r))  to  be  determinate  is  that 
"wait"  is  unplayable  for  both  players  in  at  least  one  of 

Mi,  m2. 

Proof.  Under  the  hypothesis,  val[M(a)]  is 
independent  of  a. 

The  conditions  given  so  far  are  somewhat  impractical, 
since  the  values  of  the  games  M^,  M2  are  at  least  as 
easy  to  find  as  the  optimal  strategies.  The  condition  in 
the  next  theorem  is  free  from  this  drawback,  and  it  also 
has  a  simple  heuristic  interpretation,  namely:  if  two 
actions  occur  simultaneously,  the  result  is  intermediate 
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in  value  between  the  results  of  either  action  performed 
separately. 

Theorem  4.4.  If  (4.1)  holds,  and  if,  for  all 
i  +=  0,  j  f  0,. 


(4.3)  min(ai0,  aQj )  <  a^  <  max(a.0,  aQj), 

then  M((r))  is  determinate. 


* 

Proof.  Let  i 


a0j*  = 

=  min 

j+o 

a0j* 

Let 

ai0  £ 

a0j*-' 

and 

J 

a0j  ^ 

ai*0* 

Then  by 

neither  I 

nor 

J 

Take 

jeJ 

and 

iel. 

,  j  be  such  that  a^*g  =  ®ax  a±Q > 

I  be  the  set  of  i  =)=  0  such  that 
the  set  of  j  =j=  0  such  that 
(4.1),  i*e  X  and.  j*eJ,  so  that 
nor  their  complements  are  empty. 

If  i  f  0  then  we  have 


ai0  <  a0j*  -  a0j  -  ai*0-* 

by  the  definitions  of  I,  j*,  and  J,  respectively. 
Applying  the  "betweenness"  condition  (4.3)  twice,  we 
obtain 


aij 


< 


-  0j  ^ 


<  a.  ■ 


Thus,  for  every  i£  I  (including  i  =  0) ,  a„  < 
holds  for  all  jeJ.  Similarly,  for  every  j£j,  a..  >  a..* 
holds  for  all  iel.  This  means  that  IxJ  is  a  weak  generalized 
saddlepoint  of  M(a)  (see  Sec.  3.1).  Hence  val[M(a)]  is 
independent  of  a.  Q.E.D. 
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4.4.  A  Way  of  Resolving  the  Indeterminacy 

One  may  wish  to  assign  a  formal  value  to  the 
instantaneous  game  M((r) )  when  no  true  value  exists. 

A  way  to  do  this — perhaps  the  only  "fair"  way —  is  to 
pursue  the  analogy  with  stochastic  or  recursive  games 
[16] ,  [6],  and  attempt  to  solve  the  equation 

(4.4)  a  =  val[M(a)]. 

In  effect,  we  replace  the  time  continuum  by  a  discrete, 
well-ordered  set  of  points,  making  simultaneity  after 
t  =  0  a  significant  possibility.  We  shall  wee  that 
condition  (4.1),  which  characterizes  "instantaneous" 
games,  is  intimately  related  to  the  existence  of  a  unique 
solution  to  (4.4). 

Theorem  4.5.  If  M((r))  is  an  instantaneous  game, 
then  (4.4)  has  a  unique  solution  a  =  v,  and  we  have 
V1  $  v  <;  v2. 

Proof.  Uniqueness.  Let  (4.4)  have  two  solutions 
v  <  v.  Let  x  be  optimal  in  M(v)  for  player  I  and 
let  y  be  optimal  in  M(v)  for  player  II.  Then 
xM(v)y  >  v  and  xM(v)y  <  v.  Subtracting,  we  obtain 

v  —  v  <  x(M(v)  —  M(v))y  =  XqYq  (v  —  v). 

Hence  Xq  =  y^  =  1,  and  x  and  y  are  pure.  Thus,  for 
all  i  +  0,  j  =j=  0, 
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a0j  >  v  >  v  >  aiQ. 

This  contradicts  (4-1). 

Existence.  The  function  F(a)  =  val[M(a)]  is 
monotonic  nondecreasing.  Hence,  by  (4.2),  we  have 

F (min  an.)  <  F (v-,  ) ; 
j+0  Uj  1 

in  other  words,  v-^  —  F(v^)  <  0.  Similarly,  ~  ^(v2^  >  0 
But  a  —  F(a)  is  continuous;  hence  it  has  a  zero  in 
[vl>  V2].  Q.E.D. 

Corollary.  In  the  determinate  case,  v  is  equal 
to  the  true  value. 

We  note  that  if  (4.1)  is  not  satisfied,  then  all 
values  in  the  interval  [max  a^g,  ™-n  aoj  J  are  solutions 
of  (4.4),  and  no  others. 

From  the  fact  that  F(a)  and  a  —  F(a)  are  both 
monotonic  nondecreasing  functions,  it  follows  that  the 
sequence  b,  F(b),  F(F(b)), ...  (b  arbitrary)  converges 
monotonically  to  a  solution  of  (4.4).  This  can  be  useful 
in  making  sharper  numerical  estimates;  e.g.,  we  have 
F(v^)  <  v  <  F^),  etc. 

In  the  first  indeterminate  game  of  Sec.  4.2  we  have 
v  =  0,  by  symmetry.  _In  the  second  example  we  also  have 
v  =  0;  thus,  the  first  player's  advantage,  such  as  it  is, 
disappears  if  time  is  made  discrete.  This  feature  (which 
carries  over  to  the  order— equivalent  variants)  shows 
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that  strict  inequality  need  not  hold  in  Theorem  4.5 
for  indeterminate  games. 

5.  FICTITIOUS  PLAY  1H  NON-ZERO-SUM  GAMES 

5-1.  Discussion 

The  method  of  fictitious  play  (FP)  resembles  a  multi¬ 
stage  learning  process.  At  each  stage,  it  is  assumed  that 
the  players  choose  a  strategy  that  would  yield  the  optimum 
result  if  employed  against  all  past  choices  of  their 
opponents.  Various  conventions  can  be  adopted  with  regard 
to  the  first  move,  indifferent  alternatives,  simultaneous 
vs.  alternating  moves,  and  weighting  of  past  choices. 

The  method  can  meaningfully  be  applied  to  any  finite  game, 
and  to  many  infinite  games  as  well.  (See  [3],  [4],  [5], 
pp.  82—85,  and  [14].) 

It  was  once  conjectured  that  the  mixed  strategies 
defined  by  the  accumulated  choices  of  the  players  would 
always  converge  to  the  equilibrium  point  of  the  game,  or, 
in  the  event  of  nonuniqueness,  to  a  set  of  mutually 
compatible  equilibrium  points.  This  is  the  natural 
generalization  of  Robinson's  theorem  [14]  for  the  zero— sum 
two— person  case;  it  was  recently  verified  by  Miyasawa  [12] 
for  the  special  case  of  two  players  with  two  pure  strategies 
apiece. 

The  trouble  begins,  as  we  shall  see,  as  soon  as  we 
add  a  third  strategy  for  each  player.  It  appears, 
intuitively,  that  this  size  is  necessary  to  produce  enough 
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variety;  if  FP  is  to  fail,  the  game  must  contain  elements 
of  both  coordination  and  competition.  Our  counterexamples 
include  a  whole  class  of  order— equivalent  games,  and  thus 
do  not  depend  on  numerical  quirks  in  the  payoff  matrices; 
nor  are  they  sensitive  to  the  minor  technicalities  of  the 
FP  algorithm.  It  is  clear  that  games  with  more  players, 
or  with  more  strategies  per  player,  can  exhibit  the  same 
kind  of  misbehavior. 

5.2.  A  Class  of  Nonconvergent  Examples 


We  shall  elaborate  slightly  on  the  game  described  in 
the  Introduction,  to  eliminate  any  question  of  "degeneracy." 


The  payoff  matrices  are  shown  at  right. 

Pl  a1 

We  assume  that  ai  >  b^  >  c^  and 

bl  a2  c3 

a2  P2  y2 

ai  >  ^i  >  Yi>  for  i  =  2,  3.  This 

C1  b2  a3 

y3  a3  b3 

implies,  among  other  things,  that  the  I— s  payoff  II !s  payoff 

game  is  not  constant— sum.  It  is  easy 

to  show,  by  considering  submatrices,  that  there  is  a  unique 
equilibrium  point. 

For  simplicity,  we  shall  assume  that  the  FP  choices 
are  made  simultaneously,  and  that  the  first  choice  pair 
is  TT.  Consider  any  occurrence  of  IT  in  the  FP  sequence. 

The  next  choice  of  player  I  will  certainly  be  1  again, 
since  that  strategy  has  actually  improved  in  value. 

Player  II  will  either  stay  with  1  or  shift  to  3.  Eventually, 
in  fact,  he  must  shift  to  3,  since  >  P^.  Thus,  after 
each  run  of  lT  we  will  find  a  run  of  13.  By  a  similar 
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argument,  this  will  be  followed  by  a  run  of  33,  and  then 
runs  of  j?,  "Z2,  2T,  IT,...,  in  a  never-ending  cycle. 

Suppose  that  a  run  of  TT  is  just  about  to  begin. 
Let  X  represent  the  current  'hccumulated— choices"  vector 
for  player  I  (thus,  X^  is  the  number  of  times  he  has 
chosen  row  i),  and  let  Y  be  the  same  for  player  II. 

Let  H  denote  the  current  "comparative— payoff s"  vector 
for  player  I  (thus,  =  Y^a-^  +  Y2C2  +  Y^b^,  etc.). 

Since  he  is  about  to  choose  1,  we  have 

H-^  =  max(H-^,  H3). 

Let  there  occur  r-^  choices  of  IT,  followed  by  r^3 
choices  of  13.  Then  his  new  comparative  payoffs  are 
/ 

H{  =  H1  +  rllal  +  r13b3" 

<  H2  =  H2  +  rHbl  +  r13c3 > 

H3  =  H3  +  rllcl  +  r13a3* 

V 

Since  he  shifts  to  3  at  this  point,  we  must  have 
^  Hj_.  But  H3  £  HjJ  hence  H3  -  H3  ;>  H[  -  H-p  and 
we  have 

s  al  ~  ci 

r13  £  a3  -  b3  rll’ 

Let  r33  be  the  length  of  the  3T  run  that  follows. 

By  analogous  reasoning,  we  have 

v  al  ~  Yl 

r33  ^  a3  -  P3  r13’ 
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Repeating  the  argument  four  times  more,  we  obtain 


(5.1) 


(a.  -  c.)(a. 


Yi> 


(*i  “  bi) (a±  -  p±) 


where  r  denotes  the  length  of  the  next  TT  run. 

Since  the  constant  in  (5.1)  is  greater  than  1,  the  runs 
of  11  increase  in  length  exponentially  (or  faster). 

The  same  rate  of  increase  occurs  for  the  other  choice 
pairs.  Hence  the  ratios  X^/X ^ ,  Y^/Y^ ,  i  £  j,  do  not 
converge.  In  particular,  the  normalized  strategy  vectors 
x  =  X/£  X^,  y  =  Y/S  Y.  do  not  converge  to  the  unique 
equilibrium  point  strategies. 

It  can  be  shown  that  x  and  y,  in  their  respective 
strategy  spaces,  approach  limit  cycles  which  do  not  contain 
the  equilibrium  point  strategies.  Hence  there  is  no 
convergence  to  an  equilibrium  point  even  through  subsequences. 

The  argument  we  have  given  is  independent  of  the  tie- 
breaking  rule.  With  minor  modifications  it  can  also 
handle  the  case  of  alternating  moves,  as  well  as  the  case 
of  nonintegral  run  lengths.  The  latter  implies  that  the 
differential— equation  version  of  FP  (see  [3])  will  also 
fail  to  converge  to  the  solution. 

5.3.  An  Example 

We  have  computed  the  limit  cycles  for  the  numerical 
example  shown  at  right.  The  payoffs 
do  not  satisfy  the  strict  inequal¬ 
ities  assumed  in  the  preceding  proof. 


1 

0 

0 

0 

1 

0 

0 

0 

1 

0 

0 

1 

T~ “ i 

0 

0 

Lo 

1 

0 
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and  the  constant  in  (5.1)  is  not  greater  than  1.  A 
more  refined  analysis,  however,  shows  that  the  run  lengths 
do  increase  exponentially;  in  fact,  the  ratio  of  r-j^  to 
tends  in  the  limit  to  03,  where  0  «  1.466  is  the 
real  root  of  03  _ 

The  limit  cycles  in  the  separate  strategy  spaces  are 
illustrated  in  Fig.  2.  In  the  product  space  they  form  a 
hexagon,  with  typical  vertices 

(i,  e4,  e2;  e3,  e3,  e)/c  =  •  , 

(e4,  e4,  02;  e5,  e3,  e)/ce,  etc., 

where  C  is  the  appropriate  normalizing  constant.  The 
unique  equilibrium  point  is 


(1,  1,  1;  1,  1,  l)/3  =  o. 
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